-
Notifications
You must be signed in to change notification settings - Fork 529
ci: add workflow for lambda layer publish and yank #870
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Unshure
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looked into some limitations around lambda layers, and it looks like there is a default limit of 75GB for code storage across lambda layers and lambda code. If we are bundling 8 versions of the layer per region, do we know when we are at risk of hitting this limit?
Also, can we create an issue to track adding an integ test for the deployed lambda layer?
|
|
||
| 1. **Determine target layer version**: Check existing variants to find the highest layer version | ||
| 2. **Buffer deployment**: Deploy new variants multiple times until layer version matches existing variants | ||
| 3. **Example**: If existing variants are at layer version 5, deploy new variant 5 times to reach version 5 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to buffer deployments? If we add a new variant, no one would could be depending on versions before that one. I would like to avoid backfilling versions if possible
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we don't backfill then we will need to maintain a mapping of each new variant with the version it maps to. Its mostly for a communication issue. Our docs page will be quite messy without it - but if you have an idea for how we could display that cleanly I'm all ears
With the backfill approach we just have one line item per pypi version. And can have a simple list of
"Variants with XYZ introduced in version ABC"
|
|
||
| ## Deployment Process | ||
|
|
||
| ### 1. Initial Deployment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be nice to note the exact versions of dependencies included in the layer, in case certain dependency versions come with quirks or bugs. We should also note the version of strands being released in each layer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updating to print here. Storing this long term means introducing a database of some kind. So I want to avoid the hurdles with that until we get feedback from customers that they need it
If we embed the strands version we'll be creating a new lambda layer for each pypi release rather than a new version. Thats not a pattern I've seen elsewhere
| matrix: | ||
| python-version: ${{ inputs.python_version == 'ALL' && fromJson('["3.10", "3.11", "3.12", "3.13"]') || fromJson(format('["{0}"]', inputs.python_version)) }} | ||
| architecture: ${{ inputs.architecture == 'ALL' && fromJson('["x86_64", "aarch64"]') || fromJson(format('["{0}"]', inputs.architecture)) }} | ||
| region: ${{ inputs.region == 'ALL' && fromJson('["us-east-1", "us-east-2", "us-west-1", "us-west-2", "ap-south-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "eu-north-1", "sa-east-1"]') || fromJson(format('["{0}"]', inputs.region)) }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why would we ever not deploy to all regions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If one action failed because of some networking issue. We could go back and just deploy that one so all the version are still aligned
| LAYER_NAME="strands-agents-py${PYTHON_VERSION//./_}-${ARCH}" | ||
| ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) | ||
| BUCKET_NAME="${{ env.BUCKET_NAME }}-${ACCOUNT_ID}-${REGION}" | ||
| LAYER_KEY="$LAYER_NAME/v${{ inputs.package_version }}/lambda-layer.zip" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if we need to re-release a layer version with the same strands package version? Will this key collide? Can we infer what the new layer version will be, and add that to the key name here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will overwrite, this would not impact the earlier version though.
I'll see how can I dedupe
|
|
||
| - name: Download and install package | ||
| run: | | ||
| pip install strands-agents==${{ inputs.package_version }} \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we print out and record the dependency versions for each lambda layer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updating to print here. Storing this long term means introducing a database of some kind. So I want to avoid the hurdles with that until we get feedback from customers that they need it
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
996e86f to
17a2839
Compare
Description
This pull request introduces new automated workflows for publishing and managing Strands Agents Lambda layers across AWS regions, Python versions, and architectures.
The publishing workflow, defined in publish-lambda-layer.yml, automates layer creation across 136 distinct variants, encompassing 17 regions, 2 architectures, and 4 Python versions. To prevent accidental deployments, the workflow requires explicit version confirmation through the phrase "Create Lambda Layer {version}".
For cases where layer access needs to be restricted, we've implemented a yanking workflow in yank-lambda-layer.yml.
The implementation supports 136 Lambda layers, covering combinations of Python versions (3.10, 3.11, 3.12, 3.13), architectures (x86_64, aarch64), and all 17 non opt-in AWS regions. Each layer follows the naming convention "strands-agents-py{VERSION}-{ARCH}-all" and maintains version synchronization across all variants.
Note: The review and configuration of the IAM role will be conducted outside of the GitHub workflow system to ensure proper security controls and compliance with organizational policies.
Related Issues
#636
Documentation PR
strands-agents/docs#255
Type of Change
New feature
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
Tested in dbschmigelski/sdk-python running both publishing and yanking with various configurations.
https://github.com/dbschmigelski/sdk-python/actions/runs/17740370035
https://github.com/dbschmigelski/sdk-python/actions/runs/17740080550
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.